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16.  Abstract 

This  report  describes  a  methodology  for  determining  the  effectiveness  of  the  U.S.  Coast  Guard  Marine 
Inspection  and  Boarding  Program  for  deep  draft  vessels.  Measures  of  Effectiveness  (MOEs)  were  developed  at 
the  overall  program-wide,  major  activity,  and  sub-activity  levels. 

Econometric  analysis  was  performed  on  the  relationship  between  the  number  of  personnel  and  pollution 
casualties  and  the  resource  hours  expended  by  the  Inspection  and  Boarding  programs.  The  estimates  provide 
MOEs  by  1)  quantifying  the  decrease  in  expected  number  of  casualties,  and  2)  quantifying  the  increase  in  the 
duration  in  days  to  a  casualty  that  results  from  an  increase  in  resource  hours.  A  second  methodology  called 
Risk  Based  Ranking  (RBR)  was  used  to  enumerate  the  contribution  of  factors  targeted  by  sub-activities  as 
being  key  contributers  to  the  occurrence  of  casualties. 

For  U.S.  vessels  the  results  indicate  that  resources  expended  are  effective  in  reducing  expected  number  of 
deaths,  injuries,  and  pollution  incidents.  For  foreign  vessels  the  results  are  not  robust  and  do  not  allow  clear 
inferences.  The  RBR  showed  that  the  dominant  contributors  to  maritime  risk  are  linked  to  Drills/Human 
Factors,  Steering/Navigation,  and  Cargo/Pollution  Control  sub-activity  intervention  strategies.  The  order  of 
these  factors  varies  by  vessel  service  and  country  of  registry. 

A  prototype  decision  support  system  was  developed  that  displays  the  econometric  models  graphically.  This 
report  is  issued  in  four  separate  volumes:  Volume  I  -  Executive  Summary;  Volume  II  -  Main  Report;  Volume 
III  -  Decision  Support  for  Utilizing  Measures  of  Effectiveness;  Volume  IV  -  Appendices. 
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E.l  Introduction 


The  primary  objective  of  tins  study  is  to  provide  Measures  of  Effectiveness  (MOEs)  of  the  U.S.  Coast 
Guard  Marine  Eispection  and  Boarding  Program  based  on  objective  scientific  methods.  A  secondary 
objective  of  the  study  is  to  provide  USCG  management  with  a  methodologically  and  theoretically 
sound  aid  to  eflfective  policy  decision-making.  Although  the  measures  of  effectiveness  constructed 
in  this  study  are  specific  to  the  Marine  Inspection  and  Boarding  Program,  the  methodology  of  the 
study  is  based  on  sound  theoretical  principles  that  are  probably  applicable  to  a  range  of  USCG 
activities.  Hence  the  methodology  applies  equally  to  other  important  USCG  programs  and  can  be 
similarly  used  to  measure  their  effectiveness  and  as  an  md  to  decision-making. 

Measures  of  effectiveness  are  provided  at  three  levels:  an  overall,  Program-wide  level  (Level  I),  a 
major  activity  level  (Level  H),  and  a  more  disaggregated  sub-activity  level  (Level  HI).  Level  n 
activities  cover  major  activities  including  Certificate  of  Inspections  (COI),  Reinspections,  Hull 
Drydock  Exams,  and  Foreign  Flag  Annual  Vessel  Examinations.  Level  HI  MOEs  cover  the  principal 
components  that  make  up  a  vessel  inspection  or  boarding  including  sub-activities  related  to  Cargo 
Handling,  Pollution  Control,  Steering  and  Navigation,  Document,  Drills,  Auxiliary  Equipment  Power 
Plant,  Fire-Fighting,  Hull,  and  Life  Saving.  Data  pertaining  to  deep-draft  vessel  inspections  and 
boardings  fi’om  the  Marine  Safety  Management  System  (MSMS)  database  provide  the  inputs  in  the 
construction  of  the  MOEs.  The  detailed  analyses  and  results  of  this  study  are  included  in  the  Main 
Report  -  Volume  n  and  the  Appendices  -  Volume  IV  of  this  report. 

Implementation  of  the  results  of  the  econometric  analysis  was  done  by  using  decision  support  system 
(DSS)  technology.  A  DSS  is  an  interactive,  flexible  and  adaptable  computer-based  system  that 
employs  a  model(s)  to  aid  in  problem  solving  and  decision  making.  A  key  component  of  any  DSS 
is  the  human-machine  interface;  it  should  allow  the  user  to  interact  with  the  model(s)  and  display  its 
results  in  an  easy-to-understand  format.  The  prototype  DSS  was  developed  for  the  USCG  Office  of 
Marine  Safety  on  a  PC  workstation  allowing  graphical  displays  and  interaction  with  the  model(s)  via 
a  “spreadsheet”  format,  in  Excel.  The  prototype  DSS  is  presented  in  greater  detail  in  a  companion 
document,  “Decision  Support  for  Utilizing  Measures  of  Effectiveness-Volume  HI.”  In  addition,  a 
description  is  provided  of  how  the  measures  of  effectiveness  could  be  integrated  with  a  resource 
allocation  methodology  to  support  USCG  Office  of  Marine  Safety  staff  in  budgeting,  organizational 
redesign  and  other  management  activities. 


E.2  Methodology 

Two  separate  approaches,  which  complement  each  other  nicely,  are  employed  in  this  study.  The  first 
approach  employs  state-of-the-art  econometric  models  in  the  analysis  of  a  behavioral  relationship 
between  the  number  of  Personnel  and  Pollution  casualties  and  resources  expendedhy  the  Marine 
Inspection  and  Boarding  Program.  The  ensuing  estimates  directly  provide  measures  of  effectiveness 
of  the  Program  by  (i)  quantifying  the  decrease  in  the  expected  number  of  Personnel  casualties  (deaths, 
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missing,  injured)  and  Pollution  casualties  that  result  from  an  increase  in  the  expenditure  of  such 
resource  hours,  and  (ii)  quantifying  the  increase  in  the  expected  duration-to-casualty  resulting  from 
an  increase  in  resources.  Graphical  analysis  to  ensure  integrity,  and  extensive  sensitivity  analyses  to 
test  for  robustness  of  the  econometric  results  are  performed.  The  econometric  models  are  employed 
to  provide  measures  of  effectiveness  at  the  Program,  Major  activity,  and  Sub-activity  levels. 
Measures  of  effectiveness  from  the  econometric  approach  are  broken  down  by  (i)  type  of  service,  and 
(n)  type  of  flag.  The  second  approach,  called  Risk-Based  Ranking  (RBR)  [Wheeler  (1993)],  is  used 
to  enumerate  the  contribution  of  factors,  which  are  targeted  by  Level  HI  activities  as  being  key 
contributors  to  the  occurrence  of  casualties,  to  the  risk  of  Personnel  and  Pollution  casualties.  The 
RBR  constructs  measures  of  importance  for  each  Level  HI  activity.  Large  importance  measures 
underscore  those  activities  that  may  be  expected  to  offer  the  largest  returns  from  concentrating 
resources  in  those  areas.  We  provided  risk-based  rankings  and  importance  measures  broken  down 
by  (i)  type  of  service,  (ii)  type  of  flag,  and  (iii)  MSO  and  district.  In  addition  to  being  stand-alone 
results,  the  risk-based  rankings  also  critically  supplement  the  results  of  the  econometric  analysis  and 
provide  valuable  information  in  efficient  decision-making  based  on  the  econometric  results. 


E.3  Inferences:  Marine  Inspection  Program 

Tables  E.  1  through  E.6  summarize  the  significant  results  from  the  econometric  models  developed  and 
displayed  in  the  tables  of  the  Main  Report  -  Volume  II.  The  independent  variables  are  listed  in  the 
first  column  and  dependent  variables  are  listed  in  subsequent  columns.  If  a  dependent  variable  was 
found  to  be  significant  in  at  least  one  of  the  statistical  models  developed  in  the  main  report,  then  an 
“X”  is  indicated  in  the  appropriate  box.  To  illustrate  the  interpretation  of  these  tables,  consider  Table 
E.l.  For  the  independent  variable  “Hull  Hours”  for  Passenger  Vessels,  there  is  an  X  in  the  columns 
under  dependent  variables  “Personnel  Casualties”  and  “Injuries”.  This  indicates  that  Hull  inspection 
hours  on  passenger  vessels  were  significant  in  reducing  personnel  casualties  and  injuries  for  US  Flag 
Vessels.  For  the  duration  models  such  as  Tables  E.2  through  E.4,  X’s  indicate  that  the  independent 
variable  is  significant  in  increasing  the  time  to  occurrence  of  a  casualty.  These  tables  are  a 
compilation  of  the  results  from  many  tables  and  should  not  be  misinterpreted  as  the  results  from  just 
one  statistical  model.  The  tables  in  this  volume  are  to  be  used  to  get  a  general  idea  of  the  types  of 
relationships  that  were  found  to  be  significant.  For  a  more  thorough  understanding  of  the  significance 
of  these  results,  the  specific  table  in  the  Main  Report  should  be  referenced.  These  tables  will  be 
referenced  in  parentheses  for  cross  reference  purposes  throughout  this  volume. 

From  an  analysis  of  951  U.S.  flag  deep-draft  vessels,  the  econometric  estimates  provide  clear  and 
strong  evidence  that  the  resources  expended  by  the  Marine  Inspection  Program  are  effective  in 
reducing  the  expected  number  of  deaths,  injuries,  and  pollution  incidents.  At  Level  I.  the  results 
attest  (table  4.4.1)  to  the  effectiveness  of  each  type  of  resource  hour  expended  -  hours  devoted  to  hull 
activities  (Hull  hours)  and  hours  devoted  to  machinery  activities  (Machinery  hours)  -  in  reducing 
deaths,  the  results  provide  evidence  (table  4.4.2)  of  their  effectiveness  in  reducing  injuries  especially 
on  Freight  ships  and  Tank  sWps,  and  strong  results  were  obtained  that  attest  to  their  effectiveness  in 
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reducing  pollution  occurrences  (table  4.5.2),  especially  on  Tank  ships.  The  statistical  significance  of 
the  results  is  all  the  more  stronger  if  in  interpreting  the  estimates  presented  we  recognize  that  the 
primary  objective  of  USCG  inspections  is  to  prevent  events  of  great  significance,  such  as  deaths, 
serious  injuries,  and  large  oil  spills.  In  statistical  terms,  we  consider  /-statistics  greater  than  1.00  in 
absolute  value  to  be  statistically  significant  in  assessing  effectiveness  of  the  Program.  By  this 
standard,  Hull  and  Machinery  hours  are  each  highly  effective  (table  4.5.2)  in  reducing  pollution 
occurrences  on  both  Freight  and  Tank  ships.  Hull  and  Machinery  hours  were  found  to  be  highly 
effective  in  reducing  personnel  Deaths  and  Missing  casualties  (table  4.4.1),  particularly  in  Passenger 
and  Freight  ships.  From  the  estimates  in  these  tables  we  can  quantify  the  expected  number  of 
casualties  that  may  be  reduced  by  increasing  resource  hours.  For  example,  the  estimate  of -1. 167 
for  Hull  hours  in  the  model  for  Personnel  Casualties  on  U.S.  Flag  vessels  (table  4.4.1)  implies  that 
the  expected  number  of  Deaths  and  Missing  can  be  reduced  by  .03^  (over  a  3-year  period)  if  Hull 
hours  are  increased  by  1000  (over  a  5  year  period).  The  corresponding  numbers  for  injury  reduction 
is  .12  over  a  3-year  period  (table  4.4.2),  and  for  the  reduction  in  pollution  occurrences  it  is  .29  over 
a  3-year  period  (table  4.5.2). 

At  Level  n  we  use  the  Duration  models  to  infer  how  a  change  in  the  deployment  of  resource  hours 
affects  the  time  to  a  Personnel  or  Pollution  casualty.  This  model  is  employed  because  Level  11 
activities  are  naturally  defined  in  terms  of  duration.  For  example,  a  Freight  ship  COI  is  required  once 
every  two  years.  Reinspections  (ActMty  H.  A.2)  are  required  in  alternate  years,  Hull  Exams  (Activity 
II.A.3)  are  required  twice  every  five  years  with  the  requirement  that  not  more  that  3  years  elapse 
between  two  inspections,  and  Annual  Exams  (Activity  n.B.1-3)  are  required  of  Foreign  flag  vessels 
every  year.  Hence  the  effectiveness  of  these  activities  is  best  measured  in  how  inuch  longer  they  are 
able  to  prolong  casualty-free  operation  of  a  vessel.  Results  fi-om  the  duration  analysis  given  that  a 
Certificate  of  Inspection  was  performed,  shows  Machinery  hours  and  Administrative  hours  to  be 
strongly  effective  in  increasing  time  to  Personnel  casualty,  particularly  in  Freight  ships  (tables  4.6.3 
and  4.7.3).  If  a  70%  significance  level  is  used  (/=1.00)  then  each  resource  variable  is  effective  in 
increasing  time  to  Personnel  casualty  on  vessels  of  almost  all  services.  COI's  are,  however,  not 
shown  to  be  as  clearly  effective  in  prolonging  Pollution  casualties  as  they  are  for  Personnel  casualties, 
and  only  very  weak  inferences  can  be  drawn  on  this  count.  Reinspections  are  clearly  shown  to 
prolong  duration  to  Personnel  casualties  for  all  services  (table  4.8.3),  but  the  Pollution  results  (table 
4.9.3)  do  not  allow  any  inferences  about  the  effectiveness  of  Reinspections  in  reducing  pollution 
casualties  aboard  Freight  ships  and  Tank  ships.  The  results  for  Hull  Exams  (tables  4. 10.3  and  4. 1 1.3) 
are  strong  and  impressive.  Hull  hours  are  effective  in  increasing  time  to  Personnel  casualties  in 
vessels  of  all  services,  while  Machinery  hours  are  effective  for  Freight  ships.  Hull  hours  are 
particularly  effective  in  reducing  Pollution  casualties  on  Freight  ships  and  Passenger  ships.  Clearly 
Hull  Exams  successfully  achieve  their  safety  goals  in  increasing  time  to  Personnel  as  well  as  Pollution 
casualties. 


^  Using  the  formula  for  model  #1  in  table  4.4.1  of  the  main  report,  EXP(-4.573-1.167(l)-.007(25))=  .03 
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At  Level  in  the  results  are  impressive  and  strongly  attest  to  the  efifectiveness  of  each  Level  HI  activity 
in  reducing  Personnel  and  Pollution  casualties.  A  Poisson  model  of  casualties,  similar  to  those 
employed  in  assessing  Level  I  effectiveness  of  Marine  inspection  of  U.S.  deep-draft  vessels  is 
employed  here.  Hours  devoted  to  Cargo  Handling/Pollution  Control  were  found  to  be  effective  in 
controlling  pollution  occurrences  (table  4. 12.3).  Results  for  other  Level  IH  activities  (Tables  4.13.1 
through  4.14.3)  indicate  a  great  degree  of  success  in  controlling  pollution  and  personnel  casualties. 

E.4  Inferences:  Foreign  Vessel  Boarding  Program 

The  results  from  an  analysis  of  10,904  Foreign  flag  deep-draft  vessels  are  not  as  strong  and 
unambiguous  as  those  for  U.S.  flag  deep-draft  vessels.  The  number  of  Foreign  flag  deep-draft  vessels 
examined  by  the  USCG  far  outnumbers  the  number  of  U.S.  flag  deep-draft  vessels.  But  whereas  U.S. 
flag  vessels  are  subject  to  inspection  requirements  laid  down  by  he  USCG,  and  hence  are  subject  to 
thorough  safety  checks,  this  is  not  always  true  of  Foreign  flag  vessels.  The  desired  outcome  of 
USCG  inspections  with  respect  to  Foreign  flag  vessels  is  to  minimize  Personnel  and  Pollution 
casualties  in  US.  waters,  whereas  the  objective  with  regard  to  U.S.  flag  vessels  is  to  mimmize 
casualties  anywhere.  Does  the  USCG  Boarding  Program  successfijlly  achieve  this  objective?  If  so, 
is  there  evidence  that  the  USCG  achieves  tWs  efficiently?  The  first  question  can  be  answered 
unconditionally,  simply  by  observing  the  number  of  Personnel  and  Pollution  casualties  that  have 
occurred  since  1991.  Between  1991  and  1993  there  were  61  Deaths  and  Missing,  97  Injured,  and 
535  Pollution  occurrences.  Since  well  over  10,000  deep-draft  vessels  were  examined  over  these  years 
the  casualty  rate  is  impressively  low.  Clearly,  the  USCG  program  of  examining  Foreign  flag  vessels 
is  successful  in  achieving  very  low  numbers  of  incidents  with  grave  consequences  such  as  deaths  and 
missing  and  large  pollution  casualties  in  U.S.  waters.  The  answer  to  the  second  question  is  not  easy 
to  obtain  for  two  reasons.  Compared  to  U.S.  flag  vessels,  there  is  much  more  uncertainty  about  the 
state  (i.e.,  condition)  of  a  Foreign  flag  deep-draft  vessel  that  enters  U.S.  waters  than  is  the  case  for 
a  U.S.  flag  deep-draft  vessel.  Foreign  flag  vessels  are  subject  to  widely  varying  degrees  of  monitoring 
and  regulation  depending  on  their  flag,  which  can  just  as  easily  change  from  one  port  call  to  another. 
This  makes  the  distribution  of  USCG  resource  hours  in  examining/monitoring  a  more  complicated 
and  fiizzy  task  than  it  is  in  the  case  of  U.S  flag  vessels.  We  believe  a  complete  analysis  of  the  Foreign 
flag  examination  program  must  involve  a  deep  examination  of  the  nature  of  the  monitoring^oarding 
problem.  We  feel  that  it  may  be  possible  to  design  a  ’’contract"  with  Foreign  flag  vessels  which 
enables  the  USCG  to  obtain  better  information  about  the  state  of  the  vessel  on  a  "voluntary"  basis. 
This  is  akin  to  insurance  companies  designing  contracts  in  a  manner  that  allows  them  to  "voluntarily" 
obtain  information  about  potential  insureds.  We  feel  that  it  is  possible  to  design  such  an  optimal 
contract  that  provides  the  proper  incentives  to  Foreign  flag  vessels.  According  to  the  present  system, 
the  only  incentive  a  foreign  flag  shipowner  has  to  comply  with  USCG  safety  standards  is  a  nominal 
monetary  penalty.  Hence  the  onus  is  entirely  on  the  USCG  to  detect  and  correct  problems  on  Foreign 
flag  vessels.  Naturally  the  use  of  USCG  resources  wll  not  be  as  efficient  as  in  the  case  where  the 
Foreign  flag  shipowner  "volunteers"  correct  information.  The  motivation  for  the  study  of  optimal 
incentive  contracts  is  two-fold.  Firstly,  the  provision  of  better  information  pertaining  to  a  Foreign 
flag  vessel  can  significantly  lower  the  resource  expenditure  on  such  vessels,  which  can  lead  to 
substantial  savings  given  the  large  amount  of  resource  hours  spent  by  the  USCG  on  Foreign  flag 
exams  and  boardings.  Secondly,  since  better  information  allows  a  narrower  focus,  USCG  resources 
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can  be  deployed  more  efficiently. 


At  Level  1.  It  is  shown  that  hours  spent  by  active  duty  personnel  (Regular  Hours)  and  hours  spent 
by  reserve  personnel  (Reserve  Hours)  aboard  Freight  ships  are  effective  in  reducing  Deaths  and 
Missing  (table  4.15.1),  but  these  hours  are  not  effective  in  controlling  injuries.  For  Passenger  vessels, 
Regular  hours  are  found  to  be  effective  in  controlling  injuries  (table  4. 15.2).  For  Tank  ships.  Regular 
hours  were  found  to  reduce  injuries  (table  4.15.3).  Results  indicate  that  Regular  hours  are  clearly 
effective  in  redudng  Pollution  casualties  in  vessels  of  all  services,  particularly  Freight  and  Tank  ships 
(table  4.15.4).  Various  other  model  specifications  were  estimated,  and  if  estimates  on  resource  hours 
are  compared  across  the  range  of  specifications,  we  arrive  at  the  conclusion  that  these  estimates  are 
not  robust  to  changes  in  model  specification.  Certainly  the  Foreign  flag  estimates  are  much  more 
sensitive  than  the  corresponding  U.S.  flag  results  and  do  not  allow  strong  and  unambiguous 
inferences. 

At  Level  II.  the  results  for  Personnel  casualties  are  again  sensitive  to  specification  changes.  The 
duration  analysis  of  Pollution  casualties  leads  us  to  the  conclusion  that  there  is  some  evidence, 
although  weak,  that  Regular  hours  devoted  to  Annual  Examination  of  Tank  ships  is  effective  in 
prolonging  time  to  casualty  (table  4.17.1).  But  the  same  results  based  on  specifications  where  the 
resource  hours  are  scaled  by  gross  tonnage  of  the  vessel,  show  that  Annual  Tank  ship  Examinations 
are  ineffective  in  prolonging  the  duration  to  a  pollution  casualty  (table  4.17.3),  while  no  clear 
inferences  may  be  drawn  about  Annual  Freight  ship  Examination  (table  4.17.2). 

E.5  Risk-Based  Ranking  results 

From  an  analysis  of  Freight,  Passenger,  and  Tanker  vessel  casualty  and  inspection  data  logged  in  the 
Marine  Investigations  Module  (MINMOD)  of  the  MSMS  database,  dominant  contributors  to 
maritime  risk  were  identified.  Using  MSMS  casualty  data  between  1991  and  1993,  the  risk-based 
rankings  analysis  shows  that  MSMS  MINMOD  data  do  attribute  most  risks  associated  with  Freight, 
Passenger,  and  Tanker  vessel  operations  to  casualty  causes  that  fall  under  the  purview  of  the  USCG's 
Level  in  intervention  strategies.  The  analysis  indicates  that  most  risk  is  attributed  to  causes  that  are 
linked  to  the  Drills/Human  Factors.  Steering/Navigation,  and  Cargo/Pollution  Control  Level  III 
intervention  strategies,  and  to  a  much  lesser  degree,  the  Power  Plant  and  Fire  Suppression  and 
Prevention  strategies.  The  "importance"  of  risk  that  could  not  be  attributed  to  any  of  the  Level  HI 
activities  tended  to  be  less  than  10%  of  the  risk  associated  with  the  most  dominant  Level  in 
strategies.  The  results  for  the  highest  level  of  data  aggregation  (USCG  wide)  are  summarized  in 
Tables  E.7  through  E.  12.  Table  E.9  can  be  used  as  an  example  to  interpret  these  tables.  Looking 
at  the  first  row  of  data  for  Freight  vessels,  the  1 .00  listed  under  the  “Drills/Human  Factors”  Level  in 
intervention  strategy  indicates  that  the  risk  for  the  consequence  of  “Death”  is  most  dominated  by 
Drills/Human  Factors  related  causes.  Steering/Navigation  related  causes  are  the  next  most  dominant 
factor  and  are  50%  as  important  as  Drills/Human  Factors,  followed  by  Cargo/Pollution  related  causes 
which  are  29%  as  important  as  Drills/Human  Factors,  and  so  on.  The  general  trend  for  U.S.  Freight 
and  Tanker  vessels  is  the  following:  the  most  dominant  risk  contributor  is  Drills/Human  Factors. 
followed  by  Steering/Navigation  (which  tends  to  be  only  half  as  important),  and  then  Cargo/Pollution 
Control  (approximately  a  third  as  important).  For  U.S.  Passenger  vessels  results  are  similar  except 
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that  the  rankings  of  Drills/Human  Factors  and  Steering/Navigation  are  reversed.  For  Foreign  flag 
Freight  and  Tanker  vessels  the  dominant  risk  contributor  is  Cargo/Pollution  Control,  followed  by 
Steering/Navigation  (which  is  almost  equal  in  importance),  and  then  Drills/Human  Factors  (which 
tends  to  be  approximately  only  15%  as  important).  For  Foreign  flag  Passenger  vessels  the  results 
have  Drills/Human  Factors  as  the  most  dominant  contributor  followed  by  Steering/Navigation 
(approximately  15%  as  important),  and  no  other  contributors  were  shown  to  be  more  that  5%  as 
important  as  Drills/Human  Factors.  Similar  results  can  be  inferred  for  the  District  level  analysis. 

E.6  Risk-Based  Ranking  and  Econometric  Modelling  as  Aids  to  Decision-Making 

E.6.1  Methodology 

The  risk-based  ranldng  methodology  nicely  complements  econometric  modelling  when  they  are  used 
as  decision  tools.  The  econometric  estimates  provide  an  answer  to  the  following  question:  "If  we 
increase  USCG  resource  hours  by  how  much  will  that  buy  in  reduced  personnel  and  pollution 
casualties?"  By  itself  and  without  making  further  assumptions,  however,  the  econometric  estimates 
will  not  inform  USCG  policy-makers  how  to  divide  a  fixed  pie  of  resources  among  various  activities 
eflSciently.  This  is  especially  true  i^  as  in  the  case  of  U.S.  flag  deep-draft  vessels,  many  activities  are 
shown  to  be  effective  in  reducing  personnel  and  pollution  casualties.  In  order  to  target  those 
activities  that  will  ^ve  the  largest  returns  to  an  increase  in  resource  hours,  decision-makers  can  look 
to  risk-based  ranking  measures  of  importance  to  indicate  areas  that  are  large  contributors  to  risk. 

E.6. 2  Implementing  Results 

Consider  the  econometric  results  pertaining  to  U.S.  flag  Level  HI  activities.  The  results  from  nearly 
all  the  statisical  models  attest  to  the  fact  that  an  increase  in  resource  hours  will  lead  to  substantial 
decrease  in  casualties.  However,  in  order  to  obtain  reductions  in  deaths,  injuries  and  pollution 
casualties,  many  thousand  hours  of  resources  must  be  devoted  to  these  activities,  and  given  a 
constraint  on  the  number  of  resource  hours  avrulable  to  the  USCG,  decision-makers  are  faced  with 
the  difficult  question  of  how  to  apportion  hours  efficiently  among  activities.  The  importance 
measures  from  the  risk-based  ranidngs  provide  a  natural  answer.  The  largest  increase  in  hours  should 
be  devoted  to  those  actiwties  that  cover  factors  contributing  to  the  greatest  risk  of  casualties.  For 
example,  from  Tables  E.9  and  E.ll,  increases  in  resource  hours  are  most  effective  if  devoted  to 
Drills/Human  Factors  and  Steering/Navigation  on  Freight  ships,  to  Steering/Navigation  and  Drills 
/Human  Factors  for  Passenger  ships,  and  to  Drills/Human  Factors,  Steering/Navigation,  and  Cargo 
Handling/Pollution  Control  for  Tank  ships.  We  emphasize  that  the  risk-based  ranking  importance 
measures  inform  only  about  contribution  of  factors  to  risk,  and  we  should  not  make  strong  inferences 
about  risk  reduction  or  risk  increase.  For  example,  just  because  activities  other  than  those  listed 
above  have  importance  measures  close  to  zero  does  not  mean  that  removing  resources  from  these 
activities  will  do  no  harm.  These  activities  may  have  low  importance  values  precisely  because  USCG 
inspections  are  effective  in  eliminating  the  contribution  to  the  risk  of  casualties  by  factors  covered  by 
these  activities.  We  need  to  look  to  the  econometric  estimates  to  provide  answers  about  risk 
reduction.  While  so  doing,  the  following  caveats  need  to  be  observed.  Firstly,  the  econometric 
estimates  should  not  be  extrapolated  beyond  the  range  of  data  used  for  the  sample.  Hence  if  the 
sample  does  not  contain  an  observation  (vessel)  with  zero  resource  hours  devoted  to  it,  we  cannot 
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ask  of  the  econometric  results  to  provide  an  answer  to  the  question  of  what  would  happen  to 
personnel  and  pollution  casualties  if  USCG  inspections  and  boardings  came  to  a  complete  halt. 
Econometric  estimates  are  most  reliable  for  counterfactual  experiments  that  consider  small  variations 
of  resource  hours  around  their  sample  means.  Secondly,  econometric  results  are  not  substitutes  for 
personal  judgment  and  expert  opinions  possessed  by  USCG  personnel  with  substantial  field 
experience.  Rather,  they  are  meant  to  complement  such  judgments.  Where  the  econometric  and  risk- 
based  ranking  measures  corroborate  experience  and  personal  judgement,  the  foundation  for  decision¬ 
making  is  that  much  stronger.  Where  the  data-based  measures  conflict  with  personal  judgment, 
either  personal  judgments  must  be  revised,  or  new  data  must  be  brought  to  bear  on  the  problem,  or 
both.  These  recommendations  are  deep-rooted  in  the  decision-theoretic  literature. 

E.7  The  Prototype  Decision  Support  System 

A  prototype  DSS  was  developed  to  aid  USCG  program  management  in  using  the  econometric  models 
that  generate  measures  of  effectiveness  for  the  Marine  Inspection  and  Boarding  Program.  An  easy 
to  use,  flexible  spreadsheet  format  was  employed  for  the  human-machine  interface.  Representative 
models  were  programmed  in  Excel,  the  spreadsheet  software  selected  for  the  prototype  DSS,  and 
various  displays  constructed.  The  interface  with  displays  was  assessed  by  USCG  program 
management.  Rapid  prototyping  was  used  as  the  development  process.  Extensions  to  the  prototype 
were  suggested  to  incorporate  measures  of  effectiveness  into  the  resource  allocation  decision  making 
by  USCG  program  management.  As  part  of  the  rapid  prototyping  process,  various  displays  were 
CTeated,  implemented  in  Excel  and  presented  to  the  USCG  for  review.  The  focus  of  the  effort  was 
on  models  for  determining  effectiveness  as  is  described  in  a  tutorial  companion  volume  entitled, 
"Decision  Support  for  Utilizing  Measures  of  Effectiveness  -  Volume  HI."  In  this  DSS  tutorial, 
models  fi-om  the  econometric  MOE  analysis  are  presented.  An  example  of  the  output  from  a  Poisson 
model  of  pollution  casualties  for  U.S.  Flag  vessels  is  displayed  in  Figure  E.  1  which  graphically  depicts 
the  inverse  relationship  between  resourse  hours  and  pollution  occurrences. 

Pollution  Occurrences  vs  Machine  Inspection  Hours 
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Average  Scaled  Machine  Inspection  Hours  over  a  5  year  Period 

Figure  E.l  DSS  Model  Display,  Pollution  Occurences  vs.  Machine  Inspection  Hours 


E-7 


E.  8  Recommendations  for  Further  Analysis 


1 .  Our  recommendations  for  further  analysis  of  the  effectiveness  of  USCG  activities  on  Foreign 
flag  vessels  has  to  do  with  the  quality  and  completeness  of  the  data.  As  it  stands,  the  data  provides 
information  on  only  "half'  the  experiment.  There  is  neither  data  on  the  monitoring  effort  on  Foreign 
flag  vessels  that  occurs  outside  U.S.  waters,  nor  is  there  data  on  the  occurrence  of  casualties  outside 
U.S.  waters.  We  are  thus  inferring  the  effectiveness  of  USCG  activities  based  on  a  truncated 
information  set,  and  clearly  the  inferences  will  be  much  weaker  than  those  based  on  the  complete 
information  set.  A  fruitful  line  of  enquiry  concerns  the  possibility  of  merging  the  MSMS  database 
with  Lloyds'  databases. 

2.  We  strongly  recommend  undertaking  a  study  that  analyzes  the  incentive  problems  innate  to 
the  examination  of  Foreign  flag  vessels  and  designs  an  incentive  mechanism  that  somewhat  relieves 
the  USCG  of  the  burden  of  extracting  all  (hidden)  information  about  the  state  of  the  vessel  by 
ensuring  that  vessel  operators  "voluntarily"  provide  some  information.  This  would  enable 
economically  better  and  more  efficient  decision-making  about  the  Boarding  Program,  especially  since 
a  large  proportion  of  USCG  resources  are  devoted  to  it. 

3.  The  mapping  from  inspection  types  in  the  Case  Resourse  Supplement  Table  (CRST)  of  the 
MSMS  database  into  Level  in  activities  should  be  a  more  sophisticated  fractional  mapping  which 
^ves  information  about  the  fraction  of  hours  from  a  CRST  inspection  type  that  goes  into  each  Level 
ni  activity.  It  may  be  possible  to  amend  the  CRST  inspection  types  so  that  such  a  fractional  mapping 
becomes  more  sensible.  Then  the  Level  III  analysis  of  Foreign  flag  vessel  examinations  also  becomes 
possible. 

4.  The  process  of  developing  rankings  of  inspectiorvljoarding  strategies  important  to  safety 
depends  significantly  on  the  quality  of  data  that  identifies  causal  information  and  links  it  back  to 
intervention  strategies.  The  level  of  detailed  causal  data  in  MSMS  MINMOD  is  somewhat  limited, 
and  represents  a  significant  restructuring  of  causal  data  in  the  transition  from  the  old  CASMAIN 
casualty  database  (more  comprehensive  in  this  regard)  to  the  new  MINMOD  casualty  database  (less 
comprehensive).  Our  recommendation  in  this  regard  is  therefore  that  casualty  causal  data  should  be 
redefined  in  order  to  be  informative  about  Level  HI  activities. 

5.  The  prototype  DSS  should  be  fully  implemented  by  incorporating  additional  models  into  the 
spreadsheet  format  in  Excel.  As  described  in  the  companion  volume,  three  models  were  implemented 
for  demonstration  and  assessment  purposes.  USCG  staff  should  prioritize  the  model  described  in  this 
report  and  implement  as  time  and  resources  permit. 

6.  A  “model  management”  interface  should  be  developed.  This  interface  would  guide  the  user 
to  the  model  most  appropriate  to  the  management  task  being  addressed  by  USCG  program  staff. 
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Table  £.1  Significant  Results  Summarized  from  US  Flag  Vessels 
Tables  4.1.1  -  4.5.2  in  the  Main  Report 
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Table  £.2  Significant  Results  Summarized  from  US  FlagVessel 
Duration  Models  for  COTs 
Tables  4.6.1  -  4.7.3  in  the  Main  Report 


Table  £.3  Significant  Results  Summarized  from  US  Flag  Vessel 
Duration  Models  for  Reinspections 
Tables  4.8.1  -  4.9.3  in  the  Main  Report 
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Table  £.4  Significant  Results  Summarized  from  US  Flag  Vessel 
Duration  Models  for  Hull  £xams 
Tables  4.10.1  -  4.11.3  in  the  Main  Report 
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Table  E.5  Significant  Results  Summarized  from  US  Flag  Vessels 
Poisson  Models  for  Level  HI  activities 


Independent  Variable 
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Dependent 

Variables 
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Iniuries  Incidents  Missin 
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Activity  IILl  Cargo/Pollution  Handling/Pollution  Control 
Tables  4.12.1  -  4.12.3 
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Table  E.6  Significant  Results  Summarized  from  Foreign  Flag  Vessels 
Tables  4.15.1  -  4.17.4  in  the  Main  Report 
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Table  E.7  Risk-Based  Ranking  Bin  Data  Summaiy  -  USCG  Wide  Aggregation,  U.S.  Flag 


Table  E.9  Risk-Based  Ranking  -  USCG  Wide  Aggregation,  US  Flag,  Relative  Frequency  Weighting 


Level  III  Intervention  Strategies 
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Table  E.l  1  Risk-Based  Ranking  -  USCG  Wide  Aggregation,  U.S.  Flag,  Casualty  Frequency  Weighting 
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